Benchmarking reports

from sklearn_benchmarks.report import Reporting
import pandas as pd
pd.set_option('display.max_colwidth', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

reporting = Reporting(config_file_path="config.yml")
reporting.run()

Time report

	hour	min	sec
algo
KNeighborsClassifier	0.0	13.0	52.750785
daal4py_KNeighborsClassifier	0.0	3.0	4.822835
KNeighborsClassifier_kd_tree	0.0	6.0	15.298456
daal4py_KNeighborsClassifier_kd_tree	0.0	1.0	39.911819
KMeans_tall	0.0	1.0	40.039003
daal4py_KMeans_tall	0.0	1.0	20.569132
KMeans_short	0.0	0.0	16.024887
daal4py_KMeans_short	0.0	0.0	8.350271
LogisticRegression	0.0	1.0	2.472604
daal4py_LogisticRegression	0.0	0.0	55.306440
Ridge	0.0	0.0	24.980500
daal4py_Ridge	0.0	0.0	6.617601
total	0.0	30.0	47.211386

KNeighborsClassifier vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	brute

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	n_jobs	n_neighbors	throughput	latency	accuracy_score_sklearn	accuracy_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KNeighborsClassifier	fit	0.137	0.002	1000000	1000000	100	-1	1	5.830	NaN	0.988	0.982	0.498	0.002	0.276	0.003	See
1	KNeighborsClassifier	predict	25.871	0.260	1000000	1000	100	-1	1	0.000	0.026	0.988	0.982	1.749	0.085	14.789	0.732	See
2	KNeighborsClassifier	predict	0.192	0.019	1000000	1	100	-1	1	0.004	0.000	0.988	0.982	0.095	0.001	2.021	0.203	See
3	KNeighborsClassifier	fit	0.146	0.003	1000000	1000000	100	-1	5	5.476	NaN	0.988	0.982	0.502	0.004	0.291	0.007	See
4	KNeighborsClassifier	predict	34.755	0.000	1000000	1000	100	-1	5	0.000	0.035	0.988	0.982	1.727	0.012	20.130	0.144	See
5	KNeighborsClassifier	predict	0.178	0.007	1000000	1	100	-1	5	0.004	0.000	0.988	0.982	0.093	0.001	1.909	0.078	See
6	KNeighborsClassifier	fit	0.137	0.001	1000000	1000000	100	-1	100	5.851	NaN	0.988	0.982	0.496	0.002	0.276	0.002	See
7	KNeighborsClassifier	predict	34.711	0.000	1000000	1000	100	-1	100	0.000	0.035	0.988	0.982	1.801	0.019	19.277	0.203	See
8	KNeighborsClassifier	predict	0.192	0.016	1000000	1	100	-1	100	0.004	0.000	0.988	0.982	0.094	0.001	2.039	0.168	See
9	KNeighborsClassifier	fit	0.145	0.003	1000000	1000000	100	1	1	5.509	NaN	0.988	0.982	0.509	0.007	0.286	0.007	See
10	KNeighborsClassifier	predict	13.364	0.010	1000000	1000	100	1	1	0.000	0.013	0.988	0.982	1.715	0.010	7.790	0.044	See
11	KNeighborsClassifier	predict	0.204	0.002	1000000	1	100	1	1	0.004	0.000	0.988	0.982	0.093	0.001	2.208	0.026	See
12	KNeighborsClassifier	fit	0.135	0.001	1000000	1000000	100	1	5	5.932	NaN	0.988	0.982	0.496	0.004	0.272	0.002	See
13	KNeighborsClassifier	predict	22.901	0.004	1000000	1000	100	1	5	0.000	0.023	0.988	0.982	1.714	0.006	13.359	0.049	See
14	KNeighborsClassifier	predict	0.215	0.001	1000000	1	100	1	5	0.004	0.000	0.988	0.982	0.092	0.001	2.338	0.025	See
15	KNeighborsClassifier	fit	0.136	0.001	1000000	1000000	100	1	100	5.889	NaN	0.988	0.982	0.497	0.003	0.273	0.002	See
16	KNeighborsClassifier	predict	22.916	0.006	1000000	1000	100	1	100	0.000	0.023	0.988	0.982	1.794	0.014	12.772	0.101	See
17	KNeighborsClassifier	predict	0.215	0.002	1000000	1	100	1	100	0.004	0.000	0.988	0.982	0.093	0.001	2.309	0.025	See
18	KNeighborsClassifier	fit	0.055	0.000	1000000	1000000	2	-1	1	0.290	NaN	0.988	0.982	0.097	0.003	0.567	0.017	See
19	KNeighborsClassifier	predict	22.873	0.068	1000000	1000	2	-1	1	0.000	0.023	0.988	0.982	0.258	0.001	88.682	0.325	See
20	KNeighborsClassifier	predict	0.020	0.001	1000000	1	2	-1	1	0.001	0.000	0.988	0.982	0.005	0.000	3.821	0.344	See
21	KNeighborsClassifier	fit	0.056	0.000	1000000	1000000	2	-1	5	0.286	NaN	0.988	0.982	0.098	0.003	0.572	0.016	See
22	KNeighborsClassifier	predict	33.329	0.000	1000000	1000	2	-1	5	0.000	0.033	0.988	0.982	0.260	0.002	128.313	0.912	See
23	KNeighborsClassifier	predict	0.035	0.002	1000000	1	2	-1	5	0.000	0.000	0.988	0.982	0.005	0.001	6.678	0.731	See
24	KNeighborsClassifier	fit	0.058	0.001	1000000	1000000	2	-1	100	0.278	NaN	0.988	0.982	0.097	0.002	0.594	0.012	See
25	KNeighborsClassifier	predict	33.537	0.000	1000000	1000	2	-1	100	0.000	0.034	0.988	0.982	0.307	0.002	109.392	0.559	See
26	KNeighborsClassifier	predict	0.036	0.002	1000000	1	2	-1	100	0.000	0.000	0.988	0.982	0.006	0.000	6.294	0.640	See
27	KNeighborsClassifier	fit	0.057	0.001	1000000	1000000	2	1	1	0.280	NaN	0.988	0.982	0.096	0.000	0.594	0.006	See
28	KNeighborsClassifier	predict	10.922	0.020	1000000	1000	2	1	1	0.000	0.011	0.988	0.982	0.259	0.001	42.181	0.229	See
29	KNeighborsClassifier	predict	0.016	0.001	1000000	1	2	1	1	0.001	0.000	0.988	0.982	0.005	0.000	2.958	0.258	See
30	KNeighborsClassifier	fit	0.058	0.001	1000000	1000000	2	1	5	0.278	NaN	0.988	0.982	0.097	0.001	0.593	0.008	See
31	KNeighborsClassifier	predict	21.079	0.067	1000000	1000	2	1	5	0.000	0.021	0.988	0.982	0.260	0.001	81.174	0.468	See
32	KNeighborsClassifier	predict	0.029	0.001	1000000	1	2	1	5	0.001	0.000	0.988	0.982	0.005	0.000	5.348	0.489	See
33	KNeighborsClassifier	fit	0.057	0.001	1000000	1000000	2	1	100	0.280	NaN	0.988	0.982	0.097	0.003	0.590	0.020	See
34	KNeighborsClassifier	predict	20.968	0.009	1000000	1000	2	1	100	0.000	0.021	0.988	0.982	0.308	0.002	68.071	0.528	See
35	KNeighborsClassifier	predict	0.029	0.001	1000000	1	2	1	100	0.001	0.000	0.988	0.982	0.006	0.000	5.208	0.397	See

KNeighborsClassifier_kd_tree vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	kd_tree

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	n_jobs	n_neighbors	throughput	latency	accuracy_score_sklearn	accuracy_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KNeighborsClassifier_kd_tree	fit	3.136	0.048	1000000	1000000	10	-1	1	0.026	NaN	0.986	0.985	0.684	0.007	4.587	0.084	See
1	KNeighborsClassifier_kd_tree	predict	0.454	0.005	1000000	1000	10	-1	1	0.000	0.000	0.986	0.985	0.106	0.001	4.264	0.056	See
2	KNeighborsClassifier_kd_tree	predict	0.005	0.005	1000000	1	10	-1	1	0.015	0.000	0.986	0.985	0.000	0.000	30.089	32.359	See
3	KNeighborsClassifier_kd_tree	fit	3.129	0.052	1000000	1000000	10	-1	5	0.026	NaN	0.986	0.985	0.717	0.015	4.366	0.118	See
4	KNeighborsClassifier_kd_tree	predict	0.833	0.012	1000000	1000	10	-1	5	0.000	0.001	0.986	0.985	0.196	0.002	4.244	0.076	See
5	KNeighborsClassifier_kd_tree	predict	0.003	0.001	1000000	1	10	-1	5	0.031	0.000	0.986	0.985	0.000	0.000	7.047	3.744	See
6	KNeighborsClassifier_kd_tree	fit	3.099	0.055	1000000	1000000	10	-1	100	0.026	NaN	0.986	0.985	0.691	0.005	4.484	0.085	See
7	KNeighborsClassifier_kd_tree	predict	2.605	0.026	1000000	1000	10	-1	100	0.000	0.003	0.986	0.985	0.592	0.003	4.404	0.051	See
8	KNeighborsClassifier_kd_tree	predict	0.004	0.001	1000000	1	10	-1	100	0.023	0.000	0.986	0.985	0.001	0.000	4.590	2.550	See
9	KNeighborsClassifier_kd_tree	fit	3.087	0.038	1000000	1000000	10	1	1	0.026	NaN	0.986	0.985	0.714	0.005	4.321	0.061	See
10	KNeighborsClassifier_kd_tree	predict	0.779	0.005	1000000	1000	10	1	1	0.000	0.001	0.986	0.985	0.109	0.002	7.160	0.133	See
11	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	10	1	1	0.092	0.000	0.986	0.985	0.000	0.000	4.604	3.234	See
12	KNeighborsClassifier_kd_tree	fit	3.118	0.061	1000000	1000000	10	1	5	0.026	NaN	0.986	0.985	0.694	0.007	4.491	0.098	See
13	KNeighborsClassifier_kd_tree	predict	1.467	0.008	1000000	1000	10	1	5	0.000	0.001	0.986	0.985	0.197	0.003	7.461	0.113	See
14	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	10	1	5	0.075	0.000	0.986	0.985	0.000	0.000	3.033	1.710	See
15	KNeighborsClassifier_kd_tree	fit	3.106	0.041	1000000	1000000	10	1	100	0.026	NaN	0.986	0.985	0.720	0.017	4.314	0.115	See
16	KNeighborsClassifier_kd_tree	predict	5.055	0.056	1000000	1000	10	1	100	0.000	0.005	0.986	0.985	0.596	0.007	8.489	0.136	See
17	KNeighborsClassifier_kd_tree	predict	0.002	0.001	1000000	1	10	1	100	0.041	0.000	0.986	0.985	0.001	0.000	2.602	1.276	See
18	KNeighborsClassifier_kd_tree	fit	0.780	0.011	1000000	1000000	2	-1	1	0.021	NaN	0.986	0.985	0.443	0.005	1.762	0.031	See
19	KNeighborsClassifier_kd_tree	predict	0.024	0.001	1000000	1000	2	-1	1	0.001	0.000	0.986	0.985	0.001	0.000	33.383	12.665	See
20	KNeighborsClassifier_kd_tree	predict	0.010	0.024	1000000	1	2	-1	1	0.002	0.000	0.986	0.985	0.000	0.000	89.822	223.502	See
21	KNeighborsClassifier_kd_tree	fit	0.746	0.014	1000000	1000000	2	-1	5	0.021	NaN	0.986	0.985	0.446	0.008	1.671	0.043	See
22	KNeighborsClassifier_kd_tree	predict	0.026	0.001	1000000	1000	2	-1	5	0.001	0.000	0.986	0.985	0.001	0.000	22.794	6.480	See
23	KNeighborsClassifier_kd_tree	predict	0.002	0.000	1000000	1	2	-1	5	0.008	0.000	0.986	0.985	0.000	0.000	19.713	17.380	See
24	KNeighborsClassifier_kd_tree	fit	0.747	0.006	1000000	1000000	2	-1	100	0.021	NaN	0.986	0.985	0.443	0.005	1.686	0.024	See
25	KNeighborsClassifier_kd_tree	predict	0.044	0.001	1000000	1000	2	-1	100	0.000	0.000	0.986	0.985	0.006	0.001	6.940	0.789	See
26	KNeighborsClassifier_kd_tree	predict	0.002	0.000	1000000	1	2	-1	100	0.008	0.000	0.986	0.985	0.000	0.000	16.832	12.783	See
27	KNeighborsClassifier_kd_tree	fit	0.759	0.005	1000000	1000000	2	1	1	0.021	NaN	0.986	0.985	0.444	0.005	1.709	0.024	See
28	KNeighborsClassifier_kd_tree	predict	0.023	0.001	1000000	1000	2	1	1	0.001	0.000	0.986	0.985	0.001	0.000	30.226	12.259	See
29	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	2	1	1	0.026	0.000	0.986	0.985	0.000	0.000	5.395	4.587	See
30	KNeighborsClassifier_kd_tree	fit	0.760	0.011	1000000	1000000	2	1	5	0.021	NaN	0.986	0.985	0.445	0.005	1.709	0.030	See
31	KNeighborsClassifier_kd_tree	predict	0.024	0.001	1000000	1000	2	1	5	0.001	0.000	0.986	0.985	0.001	0.000	21.826	6.525	See
32	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	2	1	5	0.025	0.000	0.986	0.985	0.000	0.000	5.409	4.516	See
33	KNeighborsClassifier_kd_tree	fit	0.745	0.008	1000000	1000000	2	1	100	0.021	NaN	0.986	0.985	0.441	0.005	1.690	0.026	See
34	KNeighborsClassifier_kd_tree	predict	0.053	0.001	1000000	1000	2	1	100	0.000	0.000	0.986	0.985	0.006	0.001	8.466	0.979	See
35	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	2	1	100	0.024	0.000	0.986	0.985	0.000	0.000	4.315	3.232	See

KMeans_tall vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	full
n_clusters	3
max_iter	30
n_init	1
tol	0.0

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	init	n_iter_sklearn	throughput	latency	adjusted_rand_score_sklearn	n_iter_daal4py	adjusted_rand_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KMeans_tall	fit	0.558	0.005	1000000	1000000	2	k-means++	30	0.861	NaN	0.001	30	0.002	0.406	0.019	1.375	0.066	See
1	KMeans_tall	predict	0.001	0.000	1000000	1000	2	k-means++	30	0.013	0.0	0.001	30	0.002	0.000	0.000	8.945	6.826	See
2	KMeans_tall	predict	0.001	0.000	1000000	1	2	k-means++	30	0.013	0.0	0.001	30	0.002	0.000	0.000	10.289	8.504	See
3	KMeans_tall	fit	0.482	0.002	1000000	1000000	2	random	30	0.995	NaN	0.001	30	0.002	0.366	0.013	1.317	0.048	See
4	KMeans_tall	predict	0.001	0.001	1000000	1000	2	random	30	0.011	0.0	0.001	30	0.002	0.000	0.000	9.379	7.514	See
5	KMeans_tall	predict	0.001	0.000	1000000	1	2	random	30	0.013	0.0	0.001	30	0.002	0.000	0.000	10.282	8.101	See
6	KMeans_tall	fit	6.138	0.142	1000000	1000000	100	k-means++	30	3.910	NaN	0.001	30	0.002	3.162	0.029	1.941	0.049	See
7	KMeans_tall	predict	0.002	0.000	1000000	1000	100	k-means++	30	0.519	0.0	0.001	30	0.002	0.000	0.000	5.291	3.035	See
8	KMeans_tall	predict	0.001	0.000	1000000	1	100	k-means++	30	0.618	0.0	0.001	30	0.002	0.000	0.000	10.452	8.210	See
9	KMeans_tall	fit	5.749	0.022	1000000	1000000	100	random	30	4.174	NaN	0.001	30	0.002	3.185	0.064	1.805	0.037	See
10	KMeans_tall	predict	0.002	0.000	1000000	1000	100	random	30	0.520	0.0	0.001	30	0.002	0.000	0.000	5.051	3.644	See
11	KMeans_tall	predict	0.001	0.000	1000000	1	100	random	30	0.629	0.0	0.001	30	0.002	0.000	0.000	8.357	5.932	See

KMeans_short vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	full
n_clusters	300
max_iter	30
n_init	1
tol	0.0

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	init	n_iter_sklearn	throughput	latency	adjusted_rand_score_sklearn	n_iter_daal4py	adjusted_rand_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KMeans_short	fit	0.242	0.012	10000	10000	2	k-means++	25	0.017	NaN	0.004	29	0.005	0.095	0.001	2.551	0.135	See
1	KMeans_short	predict	0.002	0.000	10000	1000	2	k-means++	25	0.009	0.0	0.004	29	0.005	0.001	0.000	2.650	0.639	See
2	KMeans_short	predict	0.001	0.000	10000	1	2	k-means++	25	0.013	0.0	0.004	29	0.005	0.000	0.000	10.093	7.601	See
3	KMeans_short	fit	0.107	0.000	10000	10000	2	random	30	0.045	NaN	0.004	30	0.005	0.041	0.001	2.628	0.035	See
4	KMeans_short	predict	0.002	0.000	10000	1000	2	random	30	0.009	0.0	0.004	30	0.005	0.001	0.000	2.511	0.605	See
5	KMeans_short	predict	0.001	0.000	10000	1	2	random	30	0.012	0.0	0.004	30	0.005	0.000	0.000	10.123	7.522	See
6	KMeans_short	fit	0.570	0.023	10000	10000	100	k-means++	21	0.295	NaN	0.004	25	0.005	0.324	0.014	1.759	0.104	See
7	KMeans_short	predict	0.002	0.000	10000	1000	100	k-means++	21	0.337	0.0	0.004	25	0.005	0.001	0.000	1.997	0.470	See
8	KMeans_short	predict	0.001	0.000	10000	1	100	k-means++	21	0.581	0.0	0.004	25	0.005	0.000	0.000	7.979	5.435	See
9	KMeans_short	fit	0.215	0.027	10000	10000	100	random	26	0.967	NaN	0.004	28	0.005	0.159	0.020	1.355	0.241	See
10	KMeans_short	predict	0.002	0.000	10000	1000	100	random	26	0.338	0.0	0.004	28	0.005	0.001	0.000	1.982	0.414	See
11	KMeans_short	predict	0.001	0.000	10000	1	100	random	26	0.621	0.0	0.004	28	0.005	0.000	0.000	7.715	5.010	See

LogisticRegression vs daal4py

All estimators share the following hyperparameters:

	value
penalty	l2
dual	False
tol	0.0001
C	1.0
fit_intercept	True
intercept_scaling	1
class_weight	NaN
random_state	NaN
solver	lbfgs
max_iter	100
multi_class	auto
verbose	0
warm_start	False
n_jobs	NaN
l1_ratio	NaN

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	class_weight	l1_ratio	n_jobs	random_state	n_iter	throughput	latency	accuracy_score	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	LogisticRegression	fit	11.154	0.011	1000000	1000000	100	NaN	NaN	NaN	NaN	[20]	[-0.10577568]	NaN	0.28	11.214	0.125	0.995	0.011	See
1	LogisticRegression	predict	0.000	0.000	1000000	1000	100	NaN	NaN	NaN	NaN	[20]	2.5380364415786185	0.0	0.28	0.000	0.000	0.838	0.534	See
2	LogisticRegression	predict	0.000	0.000	1000000	1	100	NaN	NaN	NaN	NaN	[20]	11.834144461224406	0.0	0.28	0.000	0.000	0.323	0.339	See
3	LogisticRegression	fit	0.756	0.015	1000	1000	10000	NaN	NaN	NaN	NaN	[26]	[2.75221518]	NaN	0.28	0.746	0.006	1.013	0.022	See
4	LogisticRegression	predict	0.002	0.000	1000	100	10000	NaN	NaN	NaN	NaN	[26]	4.875692691096868	0.0	0.28	0.003	0.001	0.561	0.154	See
5	LogisticRegression	predict	0.000	0.000	1000	1	10000	NaN	NaN	NaN	NaN	[26]	68.38682321937246	0.0	0.28	0.001	0.000	0.157	0.123	See

Ridge vs daal4py

All estimators share the following hyperparameters:

	value
alpha	1.0
fit_intercept	True
normalize	deprecated
copy_X	True
max_iter	NaN
tol	0.001
solver	auto
random_state	NaN

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	max_iter	random_state	throughput	latency	r2_score	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	Ridge	fit	0.179	0.002	1000	1000	10000	NaN	NaN	0.447	NaN	1.0	0.186	0.001	0.961	0.013	See
1	Ridge	predict	0.012	0.000	1000	1000	10000	NaN	NaN	6.514	0.0	1.0	0.020	0.001	0.625	0.033	See
2	Ridge	predict	0.000	0.000	1000	1	10000	NaN	NaN	1028.000	0.0	1.0	0.000	0.000	0.742	0.757	See
3	Ridge	fit	1.195	0.061	1000000	1000000	100	NaN	NaN	0.670	NaN	1.0	0.232	0.002	5.145	0.266	See
4	Ridge	predict	0.000	0.000	1000000	1000	100	NaN	NaN	5.080	0.0	1.0	0.000	0.000	0.670	0.603	See
5	Ridge	predict	0.000	0.000	1000000	1	100	NaN	NaN	13.326	0.0	1.0	0.000	0.000	0.618	0.703	See

Environment information

{
  "system_info": {
    "python": "3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)  [GCC 9.3.0]",
    "executable": "/usr/share/miniconda/envs/sklbench/bin/python",
    "machine": "Linux-5.4.0-1046-azure-x86_64-with-glibc2.10"
  },
  "dependencies_info": {
    "pip": "21.1.1",
    "setuptools": "49.6.0.post20210108",
    "sklearn": "1.0.dev0",
    "numpy": "1.20.2",
    "scipy": "1.6.3",
    "Cython": null,
    "pandas": "1.2.4",
    "matplotlib": null,
    "joblib": "1.0.1",
    "threadpoolctl": "2.1.0"
  },
  "threadpool_info": [
    {
      "filepath": "/usr/share/miniconda/envs/sklbench/lib/libopenblasp-r0.3.15.so",
      "prefix": "libopenblas",
      "user_api": "blas",
      "internal_api": "openblas",
      "version": "0.3.15",
      "num_threads": 2,
      "threading_layer": "pthreads"
    },
    {
      "filepath": "/usr/share/miniconda/envs/sklbench/lib/python3.8/site-packages/scikit_learn.libs/libgomp-f7e03b3e.so.1.0.0",
      "prefix": "libgomp",
      "user_api": "openmp",
      "internal_api": "openmp",
      "version": null,
      "num_threads": 2
    }
  ],
  "cpu_count": 2
}